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Authors'  Forward 

Throughout  the  course  of  this  program,  our  efforts  to  gather,  analyze,  and  compare 
interlaboratory  test  data  met  with  varying  degrees  of  skepticism  by  experts  in  the  fields  of 
penetration  mechanics  and  ballistic  testing.  The  combined  knowledge  and  experience  of 
these  scientists  and  engineers  is  impressive  indeed.  Many  of  the  participating  facilities 
have  performed  ballistic  tests  since  the  Second  World  War  and  have  developed  excellent 
reputations.  Although  the  interlaboratory  ballistic  test  series  and  resultant  data  have 
generated  a  great  deal  of  interest,  several  distinguished  members  of  the  community  have 
made  the  following  observations: 

"If  the  laboratories  don't  get  the  same  result,  then  someone  is  doing  the  test  wrong;" 
(The  Other  Guy  is  Wrong  Syndrome) 

"There  is  no  need  for  this,  our  results  compare  very  well  with  another  [single] 
laboratory;"  (Selective  Validation  Syndrome) 

"If  the  data  are  not  comparable,  then  something  must  be  wrong  with  the  test 
materials  or  the  projectiles."  (Irreprpducible  Materials  Syndrome) 

We  tried  to  address  these  and  other  concerns  insofar  as  our  resources  and  schedule 
permitted.  The  number  of  test  sites  needed  to  be  large  to  satisfy  standardized  requirements 
for  conducting  an  interlaboratory  test  and  to  justify  the  use  of  analytical  techniques.^  The 
intent  of  this  study  was  not  to  pass  judgment  on  any  specific  or  "correct"  way  to  do  a 
ballistic  test,  but  rather  to  quantify  any  existing  variability  between  laboratories.  For  this 
reason,  each  laboratory  was  assigned  a  random  number  from  1  to  9  so  the  results  could  be 
presented  anonymously,  without  a  particular  set  of  results  being  linked  to  a  particular 
laboratory.  In  this  way,  we  hoped  to  preclude  subjective  or  prejudicial  ranking  of  the 
laboratories  based  on  reputation  or  capabilities  rather  than  results.  We  also  went  to  great 
lengths  to  ensure  the  uniformity  of  both  the  armor  materials  and  projectiles.  It  is  our 
sincere  hope  that  this  work  will  serve  as  a  catalyst  for  refining  and  standardizing  ballistic 
test  methodology,  analysis,  and  data  reporting  requirements. 

Background  and  Definitions 

For  over  40  years,  scientists  and  engineers  have  grappled  with  the  problem  of  how  to 
evaluate  and  analyze  the  ballistic  performance  of  armor  materials.  Their  efforts  have 
been  complicated  by  the  large  number  of  independent  variables  associated  with  a  ballistic 
test  and  thus  the  high  costs  associated  with  ballistic  testing.  Numerous  published  papers 
address  techniques  for  reducing  and  analyzing  data.^-  *'  ^  Although  significant  efforts 
have  been  directed  at  testing  and  analyzing  ballistic  performance,  concerns  regarding  the 
reliability  and  comparability  of  ballistic  test  data  still  persist.^ 

The  statistical  value  most  closely  associated  with  ballistic  performance  is  the  value, 
defined  as  the  velocity  at  which  complete  penetration  or  partial  penetration  of  an  armor 
material  are  equally  likely  events.  There  are  several  types  of  ballistic  limits — the  Army 
Criterion,  the  Navy  Criterion,  and  the  Protection  Criterion — which  are  described  in  the 
literature.^  The  Protection  Ballistic  Limit  (PBL)  is  the  most  widely  accepted  criterion 


for  assessing  the  performance  of  lightweight  armor  materials  and  served  as  the  focus  for 
this  study.  The  experimental  outcome  of  the  PBL  ballistic  test  is  determined  by  the 
final  condition  of  a  witness  plate  placed  behind  the  armor  panel.  After  the  test,  the  witness 
plate  is  placed  in  front  of  a  light  source.  If  the  witness  plate  is  perforated  by  the  projectile 
or  spall  from  the  test  panel  (evidenced  by  light  visible  through  the  witness  plate),  that 
result  is  termed  a  complete  penetration.  If  no  perforation  is  observed  through  the  witness 
plate,  the  result  is  termed  a  partial  penetration.  Even  if  the  test  panel  is  perforated  but  the 
witness  plate  remains  intact,  the  result  is  still  defined  as  a  partial  penetration.  A  schematic 
definition  of  the  partial  and  complete  penetrations  is  shown  in  Figure  1.  The  zone  of  mixed 
results  (ZMR)  is  the  velocity  range  over  which  both  partial  and  complete  penetrations  are 
observed.  MIL-STD-662E,  V^^Ballistic  Test  for  Armor,  defines  the  Protection  Ballistic  Limit 
(PBL)  Vgp  as  follows: 

The  V50  PBL  may  be  defined  as  the  average  of  an  equal  number  of  highest 
partial  penetration  velocities  and  the  lowest  complete  penetration  velocities 
which  occur  within  a  specified  velocity  spread.  The  normal  up-and-down 
firing  procedure  is  used.  A  0.020  inch  (0.51  mm)  thick  2024  T3  sheet  of 
aluminum  is  placed  6  +  1/2  inch  (152  ±  12.7  mm)  behind  and  parallel  to  the 
target  to  witness  complete  penetrations.  Normally  at  least  two  partial  and 
two  complete  penetration  velocities  are  used  to  complete  the  BL(P).  Four, 
six,  and  ten-round  ballistic  limits  are  frequently  used.  The  maximum 
allowable  velocity  span  is  dependent  on  the  armor  material  and  test 
conditions.  Maximum  velocity  spans  of  60, 90, 100,  and  125  feet  per  second 
(ft/ s)  (18, 27, 30,  and  38  m/ s)  are  frequently  used.® 

The  main  problem  with  this  definition  is  the  extreme  latitude  it  provides  in  determining 
Vjg  value.  The  least  stringent  requirement  is  for  two  partial  and  two  complete  penetrations 
occurring  within  a  velocity  range  of  125  ft/s.  Another  variability  with  this  standard  is  the 
absence  of  standard  specimen  sizes  and  fixturing  methodology.  For  example,  tests  on 
boron  carbide  and  silicon  carbide  backed  by  Kevlar  and  Spectra  indicated  that  fixturing 
(boundary)  conditions  have  an  important  effect  on  ballistic  performance  in  a  PBL  test.^ 

PARTIAL  COMPLETE 

Penetration  Penetration 


witness  plate  is  intact  witness  plate  is  penetrated  by  projectile 

or  plate  spall 


Figure  1:  Schematic  presentation  of  partial  and  complete  penetrations. 
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Even  though  statistical  bases  for  evaluating  ballistic  data  are  established  and  documented, 
the  practical  aspects  of  conducting  a  test  do  not  often  conform  with  the  statistical 
requirements  for  analyzing  the  data.  For  example,  rudimentary  statistical  approaches  to 
data  analysis  require  at  least  30  and  preferably  45  data  points.  The  costs  associated  with 
ballistic  testing  frequently  preclude  this  large  sample  size.  Another  approach  which  relies 
on  the  method  of  maximum  likelihood  is  seldom  used  because  one  possible  result  may  be 
a  requirement  to  repeat  the  test.^  To  comply  with  cost  and  schedule  requirements,  existing 
standards  permit  the  use  of  lower  numbers  of  data  points,  typically  four  or  six  shots.  This 
approach  makes  ballistic  testing  relatively  straightforward  but  the  calculated  ballistic  limits 
have  statistically  low  confidence  levels. 

Although  work  on  analytical  techniques  continues,  the  number  of  literature  references 
which  address  the  improvements  in  experimental  techniques  used  to  generate  the  data 
for  statistical  analysis  and  modeling  is  rather  sparse.  Testing  methodology  is  an  important 
factor  because  the  models  and  analysis  can  only  be  as  good  as  the  data  on  which  they  are 
based.  In  this  respect,  our  program  is  different  from  its  progenitors  in  terms  of  its  approach. 
Where  previous  studies  have  implicitly  assumed  the  established  validity  of  ballistic  test 
data,  we  sought  to  quantify  the  comparability  of  such  data  through  interlaboratory  testing. 
Many  of  our  colleagues  have  participated  in  programs  with  two  or  three  facilities  testing 
the  same  material.  However,  those  experiments  were  not  designed  to  measure  precision 
or  reproducibility  between  the  test  facilities.  We  found  no  literature  reference  to  specifically 
designed  interlaboratory  tests  with  a  large  number  of  participants.  Even  if  two  or  three 
laboratories  could  agree  on  a  test  value,  there  is  no  statistical  foundation  for  comparing 
their  results  to  other  laboratories  conducting  ballistic  tests. 

The  test  phase  of  our  project  was  complicated  by  different  interpretations  of  the  PEL 
test:  is  it  a  systems  test  or  a  materials  test?  The  answer  to  this  question  hinges  on  how  an 
armor  material  is  defined  and  its  relation  to  an  armor  system.  An  armor  systems  test  is 
influenced  not  only  by  the  armor  material  used,  but  also  by  the  supporting  structure  and 
fixturing  technique.  A  viable  materials  property  test  should  be  reproducible  and  invariant 
within  a  prescribed  level  of  precision  A  test  for  the  velocity  at  which  the  probability  of 
partial  penetration  equals  the  probability  of  complete  penetration  for  a  given  armor  material 
should  have  this  quality.  In  other  words,  the  test  should  be  a  materials  test  and  is  not 
a  systems  test.  It  is  important,  therefore,  to  distinguish  between  ballistic  testing  of  armor 
materials  and  those  on  armor  systems. 

The  grouping  of  different  materials  to  form  laminated  armor  panels  and  spaced  arrays  is 
a  complicating  factor  in  defining  whether  these  are  armor  materials  or  armor  systems. 
Although  each  material  is  included  for  a  specific  purpose,  their  combination  may  be 
considered  a  single  entity  that  is  then  incorporated  into  a  system.  For  our  purposes,  an 
armor  system  includes  an  armor  panel  or  set  of  armor  panels,  comprised  of  one  or  more 
different  materials,  the  mechanism  used  to  fasten  those  materials  into  a  vehicle's  structure, 
and  the  structure  to  which  the  panel  is  attached.  The  fastening  mechanism  is  an  important 
component  of  the  armor  system  because  it  imposes  boundary  conditions  on  performance 
of  the  armor  materials.  These  boundary  conditions  may  have  a  profound  effect  on 
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performance  during  impact  by  kinetic  energy  penetrators.  If  the  fastening  mechanism  is 
not  designed  to  carry  the  impact  loads  associated  with  the  momentum  transfer  from  the 
projectile  to  the  armor  system,  the  fasteners  may  fail  before  the  armor  material  does.  In 
this  case,  the  system  (consisting  of  the  armor  panel,  fixturing,  and  support  structure) 
effectively  fails  even  though  the  armor  material  is  not  compromised.  We  include  the 
support  structure  in  this  definition  because  it  can  also  play  a  part  in  the  momentum  transfer 
associated  with  impact  by  a  kinetic  energy  penetrator.  This  is  an  important  point,  because 
if  the  structure  deforms  (plastically  or  elastically)  during  impact  then  it  is  contributing  to 
the  armor  system's  defeat  of  the  incident  penetrator. 

The  ballistic  performance  of  a  material  may  be  described  by  a  probability  density  function, 
(PDF),  denoted  f(x)  and  defined  by  the  following  equation^® 


j  f(x)  dx  =  \  ,  where  f(x)  >  0  and  -<»  <  x  <  <»  (1) 

Most  engineers  assume  that  tests  can  be  described  by  a  normal  (or  Gaussian)  PDF 
which  is  defined  by  the  following  function:” 

(2) 

Where  V  is  the  velocity  of  the  projectile  and  a  the  standard  deviation  from  the  (median) 
value.  For  many  materials,  this  appears  to  be  a  reasonable  assumption.  Ballistic  tests  at 
Aberdeen  Proving  Ground  (APG)  indicate  that  the  normal  cumulative  distribution  function 
(CDF)  fits  a  considerable  number  of  test  conditions.'^  Although  most  materials  seem  to 
obey  a  normal  CDF,  there  may  be  materials  for  which  other  distribution  functions 
(Lognormal,  Weibull,  Bimodal)  are  more  representative.  The  normal  CDF  is  obtained  by 
integration  of  the  normal  PDF  and  is  given  by:'^ 


CDF(V)  = 


^exp(^)d.,0<v< 


(3) 


Plots  of  the  normal  PDF  and  normal  CDF  are  displayed  in  Figures  2a  and  2b. 

Some  materials,  however,  may  follow  bimodal  or  other  distribution  functions.  For  these 
materials,  the  response  to  projectile  impact  depends  on  whether  or  not  the  projectile  shatters 
(fractures).  For  each  case  (shattered  or  unshattered  projectile),  a  separate  CDF  can  be 
used  to  describe  the  ballistic  behavior  of  the  material.  The  shatter  gap  for  the  material  may 
be  defined  as  the  velocity  difference  between  the  lower  value  for  the  unshattered  core 
CDF  and  the  higher  value  for  the  shattered  core  CDF. 
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Figure  2a:  Normal  (Gaussian)  Probability  Figure  2b:  Normal  Cumulative 

Density  Function.  Distribution  Function. 

Objective 

The  Armor  Materials  Testing  and  Data  Reporting  Standards  project  was  directed  towards 
improving  the  standards  and  procedures  associated  with  testing  armor  materials,  not  armor 
systems.  A  systems  test  implies  a  pass-fail  criterion  while  the  test  seeks  to  determine 
a  more  fundamental  materials  property.  The  test  is  confined  specifically  to  normuclear 
threats  including  warhead  fragments  (non-impulsively  loaded),  and  small  arms  kinetic 
energy  penetrators.  Some  of  the  relevant  issues  associated  with  the  V^g  test  include 
establishing  the  number  of  shots  required  to  compute  a  valid  V^g  value,  test  specimen  size, 
and  requirements  for  test  fixturing.  Specifically,  our  objectives  were  to  : 

1)  determine  the  precision  of  ballistic  test  results  between  several  Government  and 
Industrial  laboratories; 

2)  identify  possible  sources  of  bias  in  ballistic  testing; 

3)  propose  methodology  improvements  to  minimize  sources  of  bias; 

4)  validate  the  proposed  solutions. 

Procedure 

The  interlaboratory  testing  sequence  was  designed  based  on  the  American  Society  for 
Testing  and  Materials  Standard  Practice  for  Conducting  an  Interlaboratory  Study  to  Determine, 
the  Precision  of  a  Test  Method,  ASTM  Standard  E691-92.  Although  the  standard  may  be 
used  when  the  test  result  is  the  average  of  several  two-category  (go-no-go)  results, 
additional  statistical  methods  are  then  required  to  analyze  the  data.  The  standard  uses 
the  term  accuracy  to  express  the  closeness  of  a  single  test  result  to  the  "true"  or  accepted 
value.  Precision  is  a  measure  of  the  degree  of  agreement  among  several  test  results  and 
can  be  subdivided  in  terms  of  repeatability  and  reproducibility.  Repeatability  is  a  measure  of 
a  single  laboratory's  variability;  reproducibility  is  a  measure  of  variability  between 
laboratories.  Bias  is  a  measure  of  the  systematic  difference  between  a  set  of  test  results  and 
an  accepted  reference  value.^'*  In  designing  an  experiment,  the  standard  advises  as  follows: 
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To  obtain  reasonable  estimates  of  repeatability  and  reproducibility  precision, 
it  is  necessary  in  an  interlaboratory  study  to  guard  against  excessively 
sanitized  data  in  the  sense  that  only  the  uniquely  best  operators  are  involved 
or  that  a  laboratory  takes  unusual  steps  to  get  "good"  results. 

The  standard  recommends  30  or  more  laboratories  be  included  in  an  interlaboratory  study 
(ITS).  Under  no  circumstances  may  the  number  of  participants  be  less  than  6,  with  a 
minimum  of  8  required  to  begin  an  ITS.  In  our  case,  including  30  participants  was  not 
practical  from  a  financial  standpoint.  Our  study  included  9  participants,  thus  exceeding 
the  ITS  minimum. 

We  selected  participants  for  the  program  based  on  available  funding,  the  number  and 
type  of  panels  selected,  and  previous  experience  with  research  and  testing  of  light  weight 
armor  for  aircraft  applications.  Participation  of  both  Government  and  Industrial 
laboratories  was  considered  essential.  For  the  Government  laboratories,  priority  was 
accorded  to  facilities  that  have  actively  participated  in  activities  of  the  Joint  Technical 
Coordinating  Group  on  Aircraft  Survivability  (JTCG/AS)  Armor  &  Crew  Protection 
Committee.  Participation  by  Industrial  laboratories  was  solicited  through  open  bidding, 
which  does  introduce  possible  selection  bias  by  choosing  those  laboratories  offering  to  do 
the  work  and  submitting  acceptable  bids.  The  final  list  of  participants  included  research 
organizations  with  extensive  capabilities  as  well  as  facilities  devoted  to  product 
development  and  quality  control.  The  participants  included: 

Government  Laboratories 

Air  Force  Wright  Laboratories 

Army  Aviation  Applied  Technology  Directorate 

Army  Research  Laboratory,  Materials  Directorate  (Watertown) 

Naval  Surface  Warfare  Center 

Industrial  Laboratories 
Ceradyne,  Incorporated 
H.P.  White  Laboratories 
Simula  Government  Products 
Southwest  Research  Institute 
University  of  Dayton  Research  Institute 

Materials 

The  materials  selected  for  use  in  the  ballistic  tests  needed  to  be  affordable,  well 
characterized,  and  uniform.  To  this  end,  we  surveyed  published  and  unpublished  data  to 
identify  materials  for  which  data  already  existed.  Because  the  program's  sponsor  is 
dedicated  to  the  improvement  of  aircraft  survivability,  we  concentrated  on  light  weight 
materials  that  have  been  or  could  be  used  as  aircraft  armor.  Since  our  objective  was  to 
identify  difficulties  with  the  existing  military  standard  for  ballistic  testing,  we  made  a 
conscious  effort  to  identify  materials  that  would  differentiate  variations  in  the  current  test 
methodology.  Although  testing  "well  behaved"  materials  would  have  produced  useful 
information,  it  may  have  also  resulted  in  a  false  sense  of  whether  the  standard's 
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requirements  and  procedures  effect  reproducible  and  comparable  interlaboratory  ballistic 
test  data.  Our  final  consideration  was  a  fixed  budget  which  precluded  the  use  of  expensive 
ceramic  materials  such  as  boron  carbide  and  silicon  carbide.  We  could  have  selected  these 
materials,  but  only  by  reducing  the  number  of  test  samples,  thus  jeopardizing  the  statistical 
analysis. 

Based  on  these  considerations,  we  selected  two  different  types  of  panels  for  use  in  this 
study:  one  a  high  strength  steel;  the  other  a  macrocomposite  with  a  ceramic  front  face  and 
an  aramid  reinforced  fabric  back  face. 

\)  VAR  4340  Steel 

Vacuum  Arc  Remelted  (VAR)  4340  steel  is  widely  used  in  aircraft  applications  which  require 
high  strength  and  ballistic  tolerance.  Numerous  researchers  have  characterized  the  ballistic 
performance  of  this  alloy  as  a  function  of  processing  and  microstructure.  The  vacuum 
arc  remelted  (VAR)  4340  steel  used  to  fabricate  the  steel  panels  all  came  from  the  same 
heat  of  material.  The  panels  were  rolled  to  a  thickness  of  0.375  inch.  The  panels  were 
commercially  heat  treated  according  to  the  following  schedule  to  obtain  a  hardness  of 
between  50  and  52  Rockwell  C  (HRC): 

Normalize  at  1650°F  for  two  hours  and  air  cool 
Austenitize  at  1550°F  for  two  hours  and  oil  quench 
Temper  at  400°F  for  one  hour  and  air  cool 

After  heat  treatment,  0.020  inch  was  ground  from  the  face  of  each  panel  to  remove 
decarburization  and  scale  which  might  affect  the  ballistic  test  results.  ARL«MD  also  took 
microhardness  measurements  on  the  cross  section  of  some  panels  to  determine  the  extent 
of  decarburization  remaining  after  heat  treatment  and  grinding. 

21  Macrocomposite  of  AD90  Alumina  backed  by  Kevlar  Reinforced  Plastic 

The  macrocomposite  panels  consisted  of  a  single  tile  of  AD90  alumina  (Al^Og)  measuring 
5  inches  by  5  inches  by  0.535  inch  adhesively  bonded  to  an  eighteen  (18)  ply  Kevlar 
reinforced  plastic  (KRP)  panel  measuring  15  inches  square.  The  panels  were  all  fabricated 
by  a  single  vendor  to  a  single  specification. 

Data  from  previous  ARL*MD  tests  indicated  that  ceramic /composite  armor  panels  can 
have  markedly  different  performance,  based  upon  the  thickness  ratio  of  front  face  to  back 
face  component. Test  data  from  other  research  indicated  that  similar  armor  panel 
configurations  produced  a  large  zone  of  mixed  results  due  to  unexplainable  high  velocity 
partial  penetrations.^®  There  are  other  examples  of  this  behavior  where  the  difference 
between  the  reported  value  and  the  highest  partial  penetration  velocity  varied  by  400 
fps  in  one  case  and  538  fps  in  another  case.^^'^®  Although  these  studies  did  not  produce 
enough  data  to  define  the  cumulative  distribution  function,  they  clearly  indicate  the 
propensity  of  ceramic  faced,  macrocomposite  armor  panels  to  exhibit  anomalous  behavior. 
In  other  words,  for  some  configurations  it  is  possible  to  calculate  two  different  V^^  values 
for  two  different  velocity  regimes.  The  participants  in  the  interlaboratory  tests  were  not 
informed  of  this  behavioral  characteristic  of  the  macrocomposite  panels  in  advance  of  the 
tests. 
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Ballistic  Testing  Methodology 

In  deciding  how  many  samples  each  laboratory  should  test,  we  desired  to  have  a  quantity 
of  panels  sufficient  to  permit  valid  statistical  evaluation,  recognizing  a  requirement  for 
too  many  panels  would  render  a  ballistic  test  unaffordable  for  some  materials.  We  decided 
that  three  (3)  steel  panels  capable  of  sustaining  eight  or  nine  shots  each  and  sixteen  (16) 
ceramic /Kevlar  macrocomposite  panels  capable  of  sustaining  a  single  shot  represented 
an  effective  trade  off  between  available  resources  and  the  requirements  for  statistical 
analysis. 

ARL*MD  provided  0.50  caliber  AP  M2  bullets  and  primed  cases  to  each  participant.  We 
randomly  selected  the  bullets  from  ARL*MD's  bunker;  this  is  also  the  source  of  bullets 
used  to  produce  data  contained  in  Ballistic  Technology  of  Lightiveight  Armor  AMMRC  TR 
81-20 — also  known  as  the  AMMRC  Armor  Handbook.  Even  though  the  largest  single 
source  of  small  arms  ballistic  data  in  the  United  States  was  developed  with  these  rounds, 
we  characterized  the  average  hardness  for  the  bullets  by  randomly  selecting  and  testing 
ten  (10)  bullets  for  Rockwell  C  Hardness  tests.  Each  laboratory  was  responsible  for 
providing  powder,  a  target  fixture,  barrel  with  firing  mechanism,  and  velocity  screens. 

We  selected  a  starting  velocity  of  1600  feet  per  second  (fps)  for  both  the  steel  panels  and 
the  macrocomposite  panels.  The  test  director  of  each  laboratory  selected  all  subsequent 
velocities.  The  obliquity  for  all  tests  was  0°.  We  did  not  specify  any  methodology  for  the 
test  engineer  to  use  when  selecting  the  next  velocity,  directing  only  that  each  test  director 
use  MIL-STD-662E.  All  of  the  laboratories  used  variants  of  the  up  &  down  method  to  select 
projectile  velocities."^  This  technique  is  based  on  a  bisection  algorithm.  The  advantage  of 
this  approach  is  that  relatively  few  shots  are  required  before  a  value  can  be  calculated. 
Most  of  the  participants  focused  their  attention  in  the  velocity  range  from  1400  to  2000  fps 
for  both  target  types.  Only  when  the  test  engineer  was  confident  that  the  PEL  had 
been  determined  did  the  ARL*MD  representative  select  velocities  for  the  remaining 
(usually  two  to  four)  shots. 

There  are,  however,  disadvantages  to  the  up  &  down  approach  to  velocity  selection.  The 
main  disadvantage  is  that  it  often  precludes  exploring  velocity  regimes  above  and  below 
the  apparent  value.  Because  the  test  engineer  selects  the  powder  charge  (and  thereby 
the  projectile  velocity),  the  apparent  value  calculated  from  the  test  data  is  inherently 
biased.  The  up  &  down  method  assumes  a  normal  distribution  with  a  narrow  zone  of 
mixed  results  (ZMR).  If  these  conditions  are  not  met,  then  the  method  will  not  always 
produce  valid  results.  The  up  &  down  method  produces  a  single  value  and  will  not 
help  identify  the  presence  of  any  shatter  gap.  Finally,  the  up  &  down  method  tends  to 
produce  apparent  values  which  are  either  higher  or  lower  than  the  actual  Y^  value.  If 
the  velocity  of  the  first  shot  is  near  the  actual  value,  MIL-STD-662E  requires  the  second 
shot  be  fired  at  least  100  fps  higher  or  lower  than  the  actual  value.  Whenever  the 
second  and  third  shots  result  in  a  low  probability  result  (a  complete  penetration  below  the 
actual  Vgp  value  or  a  partial  penetration  above  the  actual  value),  the  apparent  value 
will  be  driven  still  further  from  the  actual  value. 
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Residual  velocities  of  the  bullet,  fragments,  and  spall  were  not  measured.  Although  residual 
velocities  are  important  data  for  vulnerability  studies,  they  add  an  increment  of  cost  to 
ballistic  testing  which  was  beyond  the  resources  of  this  program.  Moreover,  if  results 
from  many  laboratories  do  not  produce  a  consensus  value  for  a  particular  type  of 
armor  panel,  residual  velocity  measurements  cannot  possibly  agree  either.  For  that  reason, 
we  elected  to  focus  this  program  on  characterizing  projectile  target  interactions  and  defer 
questions  related  to  residual  velocity  measurement  and  behind  the  armor  effects  to  the 
future. 

Test  Results 

Each  laboratory  provided  the  Project  Manager  with  a  report.  Although  each  report  had  a 
unique  presentation  format,  they  all  included  a  table  listing  the  shot  order,  target  type, 
impact  velocity,  and  result  (partial  or  complete  penetration).  We  randomly  assigned  each 
laboratory  a  number  between  1  and  9,  so  that  the  results  could  be  displayed  anonymously. 
We  collated  the  results  based  on  the  type  of  target  and  the  order  in  which  the  shots  were 
taken.  Table  1  lists  the  raw  velocity  data  for  the  steel  panels  in  feet  per  second  (fps);  Table 
2,  for  the  macrocomposite  panels.  Since  each  laboratory  received  three  steel  panels,  the 
results  are  presented  for  each  panel.  In  addition.  Tables  3  and  4  show  the  velocity  increment 
from  one  shot  to  the  next  for  the  steel  and  macrocomposite  panels,  respectively.  MIL- 
STD-662E  typically  recommends  a  velocity  increment  or  decrement  of  50  fps  or  100  fps. 

The  data  from  each  laboratory  were  combined  to  form  a  Grand  Summary  for  each  type  of 
target  panel.  This  result  is  displayed  graphically  in  Figure  3.  For  the  steel  panels,  the 
difference  between  the  highest  partial  penetration  and  the  lowest  complete  penetration  is 
685  fps;  for  the  macrocomposite  panels,  859  fps.  Figure  4  is  a  graphical  representation  of 
the  penetration  data  for  the  steel  panels  in  the  velocity  range  of  1400  to  2000  fps;  Figure  5, 
for  the  macrocomposite  panels  in  the  velocity  range  of  1500  to  2000  fps.  The  numbers 
above  and  below  the  graphs  indicate  the  number  of  partial  and  complete  penetrations 
above  and  below  the  velocity  range.  The  velocity  ranges  for  Figures  4  and  5  encompass 
approximately  90%  of  all  the  data  points.  The  two  figures  are  included  to  show  graphically 
the  comparability  of  the  raw  data  (partial  and  complete  penetration  velocities). 

Some  of  the  steel  panels  were  penetrated  at  velocities  below  1500  fps.  This  runs  contrary 
to  published  results  for  this  material  with  a  hardness  of  between  50  and  52  HRC  at  the 
areal  density  used  for  this  series  of  tests.  To  determine  if  decarburization  was  present, 
and  if  so  to  what  depth,  ARL«MD  sectioned  some  of  the  panels  and  performed 
microhardness  tests.  The  results  of  these  tests  indicate  that  no  more  than  0.015  inch  of 
"soft"  (less  than  48  HRC)  material  remained  on  the  panel  surface. 
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Table  1.  Summary  of  Steel  Velocity  (ft/s)  Data  in  Shot  Sequence  Order. 


Table  2.  Summary  of  Macrocomposite  Velocity  (ft/s)  Data  in  Shot  Sequence  Order. 
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Table  3.  Velocity  (ft/ s)  Delta  for  Steel  Panels. 
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Alumina /KRP 
Partial 

^  Alumina /KRP 
Complete 


VAR  4340  Steel 
Partial 

^  VAR  4340  Steel 
Complete 


Figure  3:  Grand  Summary  of  Partial  Penetration  and  Complete  Penetration  Data. 


Analysis 

Our  analysis  of  the  interlaboratory  data  is  intended  to  determine  whether  the  results  of 
ballistic  testing  at  various  test  facilities  are  reproducible.  We  did  not  try  to  quantify 
parameters  such  as  bias  because  this  requires  comparison  to  a  value  accepted  as  "true". 
Based  on  our  initial  review  of  the  data,  there  was  no  compelling  basis  for  identifying  any 
laboratory's  result  as  being  the  "true"  value.  Indeed,  at  the  completion  of  the  testing 
sequence,  each  laboratory  expressed  confidence  that  they  had  indeed  determined  the  "true" 
V50  value. 

For  a  test  method  to  be  determined  reproducible,  it  must  be  possible  to  have  the  test 
performed  on  the  same  material,  at  various  facilities,  and  achieve  results  that  are  not 
statistically  different  with  a  reasonable  confidence  level.  We  minimized  or  eliminated 
many  material  variables  inherent  in  the  conduct  of  a  ballistic  test  by  providing  each  lab 
with  statistically  identical  target  panels  and  0.50  caliber  AP  M2  projectiles.  We  instructed 
the  laboratories  to  conduct  the  ballistic  test  in  accordance  with  MIL-STD-662E. 

The  Project  Manager  made  no  attempt  to  control  each  laboratory's  test  fixtures,  velocity 
measuring  equipment,  environmental  conditions  or,  most  significantly,  calculation 
methods.  We  assumed  that  any  variations  in  the  test  results  were  attributable  to  these 
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Figure  4:  Graphical  Representation  of  Data  for  the  Steel  Panels. 
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Lab  6  Incomplete  o  Lab  7  Incomplete  o  Lab  8  Incomplete  O  Lab  9  Incomplete 


Figure  5:  Graphical  Representation  of  Data  for  the  Macrocomposite  Panels 


Lab  6  Incomplete  o  Lab  7  Incomplete  o  Lab  8  Incomplete  o  Lab  9  Incomplete 


variables.  We  conjecture  that  the  most  significant  source  of  variability  results  from  the 
method  used  to  select  shot  velocities  and  thus  calculate  the  value.  Most  laboratories 
rely  upon  experience  and  intuition  when  selecting  velocities.  Although  most  laboratories 
used  variants  of  the  up  &  down  method  for  velocity  selection,  none  of  them  applied  a 
single  velocity  selection  algorithm  uniformly.  We  will  use  our  analysis  of  the  test  data  to 
draw  conclusions  about  the  suitability  of  such  heuristic  test  methods,  in  combination  with 
any  non-standardized  fixturing  hardware. 

We  focused  our  attention  on  the  laboratory  reports,  searching  for  an  acceptable  method 
for  comparing  the  results.  The  reports  returned  from  the  laboratories  participating  in  the 
interlaboratory  study  included  raw  firing  data,  and  in  most  cases  a  reported  value. 
They  differed  in  format  and  completeness.  Some  laboratories  reported  separate  values 
for  each  steel  panel.  Others  reported  one  value,  calculated  using  the  shots  against  all 
of  the  three  steel  panels,  collectively.  Some  laboratories  reported  a  value  based  on  the 
average  of  the  values  obtained  from  each  respective  panel. 

Deciding  upon  a  method  to  compare  the  results  obtained  from  each  lab  is  nontrivial  due 
to  the  variation  in  the  reports  and  the  method  employed  to  calculate  a  value.  The 
laboratories  reported  the  firing  sequence,  velocity  and  test  result,  for  each  test  shot, 
uniformly  well.  Consequently,  it  appears  reasonable  to  use  these  elements  of  the  raw  data 
to  recalculate  values  using  a  standard  procedure.  We  chose  to  recalculate  the  values 
strictly  using  Test  Operations  Procedure  (TOP)  2-2-710  procedures. 

TOP  2-2-710,  Ballistic  Tests  of  Armor  Materials,  describes  several  different  methods  for 
calculating  a  value.^  Each  method  is  characterized  by  the  number  of  partial  and 
complete  penetrations  and  the  maximum  velocity  range  encompassing  the  data  points 
used  to  calculate  the  value.  Some  of  the  methods  are  listed  in  Table  5. 

The  first  four  methods  are  based  on  up  &  down  velocity  selection.  The  first  round  is 
prepared  to  yield  a  firing  velocity  nearly  equal  to  the  expected  value.  The  second 


Table  5:  Calculation  Methods  and  Velocity  Ranges. 


V50  Type 

Velocity  Range 

Assumed 

Distribution  Type 

1  partial  &  1  complete 

15  m/s  (50  fps) 

Normal 

2  partials  &  2  completes 

18  m/s  (60  fps) 

Normal 

3  partials  &  3  completes 

27, 38,  or  46  m/s  (90, 125, 150  fps) 

Normal 

5  partials  &  5  completes 

38, 46  m/s,  or  unlimited  (125, 150  fps) 

Normal 

Langlie  (12  rounds) 

not  stipulated 

Normal 

Sampling-of-Levels 

single  fixed  velocity 

Not  Normal 

Probit  Design 

several  fixed  velocities 

Normal 
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round  is  prepared  to  yield  a  firing  velocity  ±100  fps  relative  to  first  round.  If  the  result  of 
the  first  round  results  in  penetration  of  the  target,  the  lower  velocity  is  used;  if  the  round 
failed  to  penetrate  the  target,  the  higher  velocity  is  used.  Subsequent  rounds  are  prepared 
±100  fps  for  same  result  or  ±50  fps  for  a  reversal.  After  the  first  reversal,  subsequent 
rounds  are  prepared  ±50  fps  for  same  result  or  ±25  fps  for  an  additional  reversal.  Note 
that  the  Langlie  Method  requires  application  of  the  method  of  maximum  likelihood  for 
data  reduction. 

We  organized  the  firing  data  from  each  lab  in  the  order  the  shots  were  taken.  A  value 
was  calculated  considering  each  successive  shot  in  the  following  manner: 

(1)  When  one  partial  penetration  (PP)  and  one  complete  penetration  (CP)  result 
within  a  maximum  velocity  spread  of  50  fps,  the  arithmetic  mean  of  the  two 
velocities  is  reported  as  the  V50  value.  The  velocity  spread  is  calculated  by 
subtracting  the  lowest  velocity  of  the  two  shots  from  the  higher  velocity  in 
the  sample. 

(2)  When  two  PPs  and  two  CPs  result  within  a  maximum  velocity  spread  of  60 
fps,  the  arithmetic  mean  of  the  four  velocities  is  reported  as  the  V50  value. 

The  velocity  spread  is  calculated  by  subtracting  the  lowest  velocity  in  the 
sample  from  the  highest  velocity  in  the  sample. 

(3)  When  three  PPs  and  three  CPs  result  within  a  maximum  velocity  spread  of 
150  fps,.the  arithmetic  mean  of  the  six  velocities  is  reported  as  the  V50  value. 

The  velocity  spread  is  calculated  by  subtracting  the  lowest  velocity  in  the 
sample  from  the  highest  velocity  in  the  sample. 

(4)  When  five  PPs  and  five  CPs  result,  the  arithmetic  mean  of  the  ten  velocities 
is  reported  as  the  V50  value.  The  velocity  spread  is  calculated  by  subtracting 
the  lowest  velocity  in  the  sample  from  the  highest  velocity  in  the  sample. 

Table  6  lists  the  values  calculated  as  described  above,  for  the  steel  panels.  Table  7 
contains  similar  information  for  the  macrocomposite  panels.  The  tables  also  present  the 
Vgg  values  reported  by  each  laboratory.  Having  calculated  the  values  using  a  single, 
uniform  methodology,  we  can  now  proceed  to  compare  the  results  obtained  from  each  lab 
using  statistical  analysis.  Recalculating  the  data  is  an  important  step.  We  needed  to 
normalize  the  data  to  establish  a  uniform  basis  for  statistical  comparison.  Using  this 
approach,  excessive  variation  in  the  values  can  be  attributed  to  a  combination  of  the 
velocity  selection  heuristic,  and  the  test  hardware  used. 

The  data  in  each  row  of  Tables  6  and  7  can  be  compared  using  a  variety  of  statistical 
comparison  techniques.  One  particularly  well  documented  comparison  method  is  the 
Tukey  Method.  The  Tukey  method  conducts  pairwise  comparisons  of  n  sample  means  to 
establish  whether  |i.  =  p.,  for  /,  y  =  l...n,  at  a  selected  level  of  confidence.  We  can  use  the 
Tukey  Method  to  conduct  multiple  pairwise  comparisons  of  the  recalculated  values 
displayed  in  Tables  6  and  7.  The  method  allows  the  analyst  to  select  a  confidence  level  at 
which  to  compare  the  means  from  various  samples  of  equal  size.  In  our  case  the  means  in 
question  are  the  values.  When  the  sample  sizes  producing  the  means  are  equal  and  all 
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Table  6:  Table  of  Values  for  Steel  Panels. 


V50  Type 

Lab  1 

Lab  2 

Lab  3 

Lab  4 

Lab  5 

Lab  6 

Lab  7 

Lab  8 

Lab  9 

1  &  1  in  50  fps 

1813 

1820 

1807 

1657 

1595 

1258 

1838 

1583 

1449 

2  &  2  in  60  fps 

1795 

1852 

1823 

1684 

1562 

1677 

1791 

1598 

1429 

3  &  3  in  150  fps 

1773 

1850 

1878 

1657 

1556 

1644 

1794 

1602 

1450 

5&5 

1815 

1834 

NC* 

1693 

1503 

1514 

1746 

1585 

NC* 

Reported 

1819 

1876 

1867 

1683 

1497 

1520 

1716 

1602 

1429 

Table  7:  Table  of  Values  for  Macrocomposite  Panels. 


V50  Type 

Lab  1 

Lab  2 

Lab  3 

Lab  4 

Labs 

Lab  6 

Lab  7 

Lab  8 

Lab  9 

1  &  1  in  50  fps 

1704 

1896 

1601 

1638 

1774 

1876 

1658 

1671 

1673 

2  &  2  in  60  fps 

1747 

1781 

1772 

1684 

1755 

1897 

1652 

1675 

1689 

3  &  3  in  150  fps 

1741 

1845 

1804 

1695 

1761 

1906 

1634 

1669 

1692 

5&5 

1760 

1975 

NC* 

1706 

1758 

1872 

1657 

1711 

1908 

Reported 

1760 

1814 

1790 

1694 

1758 

1841 

1657 

1708 

1692 

NC  stands  for  Not  Calculated,  indicating  insufficient  data  to  calculate  the  V50  value. 


pairwise  comparisons  are  to  be  made,  the  Tukey  method  is  preferred  over  other  comparison 
techniques  because  it  yields  the  narrowest  confidence  intervals.^'  The  entries  in  each 
respective  row  of  Tables  6  and  /--except  the  reported  V^^  row)— are  the  result  of  the  arithmetic 
mean  of  equal  sized  samples. 

We  selected  several  confidence  levels  for  use  in  our  analysis.  The  confidence  levels  we 
selected  include  0.01  (99%),  0.05  (95%),  0.10  (90%),  0.15  (85%)  and  0.25  (75%).  Lowering 
the  confidence  level  increases  the  likelihood  of  rejecting  the  hypothesis  that  |i.  =  p..  In 
other  words,  we  presume  initially  that  the  mean  (V^^)  values  are  equal.  We  want  a  certain 
level  of  confidence  in  our  determination  of  any  inequalities.  At  lower  confidence  levels, 
we  expect  more  pairwise  comparisons  to  result  in  a  determination  of  significant  difference 
because  the  difference  between  the  two  means  being  considered  need  not  be  as  great  to 
arrive  at  the  conclusion  that  they  are  unequal.  At  higher  confidence  levels  fewer  differences 
are  detected,  but  those  detected  are  more  significantly  different.  We  used  a  broad  spectrum 
of  confidence  levels  to  demonstrate  that  even  at  high  levels  of  confidence  a  substantial 
percentage  of  significant  differences  exists  between  the  V^^  values  from  the  interlaboratory 
test  data. 

We  selected  Minitab  a  widely  available  statistical  software  package,  to  aid  in  the  analysis 
because  it  performs  the  Tukey-Method  calculations  quickly  and  provides  the  output  in  an 
easily  understood  manner. For  example,  the  data  in  row  1  of  Table  6  gives  V^^  values 
calculated  using  one  partial  and  one  complete  penetration  within  a  velocity  range  of  50 
fps,  for  each  lab.  Consequently,  the  sample  size  for  each  V^^  value  is  two.  Minitab  was 
used  to  do  the  Tukey  analysis  at  the  0.05  (95%)  confidence  level.  Figure  6  displays  the 
resulting  output.  Each  cell  in  the  Table  of  Pairwise  Comparisons  represents  a  range  of 
values  that  bounds  the  possible  difference  between  the  V^^  value  from  lab.  and  lab.  at  the 
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Figure  6:  Table  of  Pairwise  Comparisons,  Minitab  Output  for  0.05  Confidence  Level 


LAB  1 

2 

3 

4 

5 

6  7  8 

LAB 

2 

-158.9 

Example:  The  difference  between  the  V50 

43.9 

reported  by  lab3  and  lab4  is  between  49 

and 

252  with  95%  confidence.  The  range 

3 

-95.9 

-38.4 

does 

not  include  zero,  so  the  two  V50's 

106.9 

164.4 

are 

statistically  different. 

4 

54,6 

112.1 

49.1 

257,4 

314.9 

251.9 

Tukey' s  pairwise  comparisons 

Family  error  rate  =  0.0500 

5 

116,1 

173.6 

110.6 

-39.9 

Individual  error  rate  =  0.00336 

318,9 

376.4 

313.4 

162.9 

6 

453,1 

510.6 

447.6 

297.1 

235.6 

655.9 

713.4 

650.4 

499.9 

438.4 

7 

-126.4 

-68.9 

-131.9 

-282.4 

-343.9 

-680.9 

76.4 

133.9 

70.9 

-79.6 

-141.1 

-478.1 

8 

127.6 

185.1 

122.1 

-28.4 

-89.9 

-426.9  152.6 

330.4 

387.9 

324.9 

174.4 

112.9 

-224.1  355.4 

9 

262.6 

320.1 

257.1 

106.6 

45.1 

-291.9  287.6  33.6 

465.4 

522.9 

459.9 

309.4 

247.9 

-89.1  490.4  236.4 

0.05  confidence  level.  If  the  interval  does  not  contain  zero  there  is  a  significant  statistical 


difference  between  the  two  values  from  the  laboratories  being  compared.  Italicized 
entries  in  the  table  indicate  cells  where  a  significant  difference  exists.  We  can  use  the  same 
procedure  to  compare  the  data  in  each  row  of  Table  6  and  Table  7  at  the  significance  levels 
we  selected.  We  performed  a  Tukey  analysis  for  each  type  of  value  and  each  of  the  two 
panel  materials.  The  Appendix  includes  the  results  of  these  analyses  at  the  0.05  confidendce 
level  in  the  following  tables: 

Table  Al:  Steel  Values  (1  PP,  1  CP)  In  50  fps 
Table  A2:  Steel  V3g  Values  (2  PP,  2  CP)  In  60  fps 
Table  A3:  Steel  V^^,  Values  (3  PP,  3  CP)  In  150  fps 
Table  A4:  Steel  V^^  Values  (5  PP,  5  CP) 

Table  A5:  Steel  V3j,  Values  (1  PP,  1  CP)  In  50  fps 
Table  A6:  Ceramic  V3„  Values  (2  PP,  2  CP)  In  60  fps 
Table  A7:  Ceramic  Values  (3  PP,  3  CP)  In  150  fps 
Table  A8:  Ceramic  V3jj  Values  (5  PP,  5  CP) 


The  total  number  of  pairwise  comparisons  possible  using  the  standardized  data  in  Tables 
6  and  7  is  265.  The  results  of  our  Tukey  analyses  are  given  in  the  Table  8. 

At  the  0.01  (99%)  confidence  level,  149  of  the  265  possible  pairwise  comparisons  are  found 
to  have  a  significant  statistical  difference.  Restated,  the  Tukey  analysis  indicates  that  56% 
of  all  pairwise  comparisons  of  lab  V3Q  values  show  a  significant  statistical  difference  at  our 
most  discriminating  level  of  significance. 
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Table  8;  Results  of  Tukey  Analysis  for  Five  Levels  of  Confidence 


Confidence  Level 
(%) 

Significant 

Differences 

Percent  of  total 
Comparisons  (%) 

99 

149 

56 

95 

162 

61 

90 

169 

64 

85 

172 

65 

75 

181 

68 

In  presenting  the  Tukey  analysis,  we  must  keep  in  mind  that  "there  is  more  controversy 
among  statisticians  regarding  which  multiple  comparisons  procedure  to  use  when  sample 
sizes  are  unequal  than  when  sample  sizes  are  equal."  The  method  recommended  in  Beyond 
ANOVA:  Basics  of  Applied  Statistics'^  is  a  modified  Tukey  method  used  when  sample  sizes 
vary,  but  are  'reasonably  close'.  The  entries  in  the  reported  row  in  each  table  represent 
the  arithmetic  mean  of  unequal  sample  sizes.  Unfortunately,  sample  sizes  varied  from  4 
to  22,  and  in  some  cases  were  not  reported  at  all!  Consequently,  the  Modified  Tukey 
method  may  not  be  an  acceptable  comparison  technique  for  this  data.^‘* 

In  addition  to  the  Tukey  analysis,  we  also  applied  a  modified  Kniskal  -  Wallis  (KW)  analysis. 
Like  the  Tukey  analysis,  the  KW  analysis  assumes  that  the  samples  are  randomly  and 
independently  selected.  Unlike  the  Tukey  analysis,  the  KW  analysis  does  not  assume  an 
underlying  distribution.  However,  it  requires  at  least  five  data  points,  which  precludes 
consideration  of  all  sets  of  1  &  1  (two  data  points)  and  2  &  2  (four  data  points)  V^g  values. 

The  result  of  the  KW  analysis  is  a  test  statistic  K,  which  is  defined  as  follows: 


12  V  Bl 
n  (n  +  1 )  ,4l  «, 


3(n+  1) 


(4) 


This  test  statistic  is  used  to  determine  if  one  laboratory  is  statistically  different  from  the 
other  laboratories  as  a  group.  This  is  an  important  difference  from  the  Tukey  analysis 
which  permits  one  to  one  comparison  between  the  laboratories.  The  results  of  this  analysis 
are  shown  in  Table  9.  These  K  values  are  then  compared  to  those  in  Table  10  for  the 
appropriate  confidence  level  and  degrees  of  freedom.  The  degrees  of  freedom  is  simply 
the  number  of  laboratories  less  one.  Although  the  KW  analysis  method  requires  fewer 
underlying  assumptions  and  only  provides  comparison  of  each  laboratory  to  the  group, 
the  results  lead  us  to  the  same  conclusion  as  the  Tukey  analysis,  namely  that  each  individual 
laboratory  cannot  be  compared  statistically  to  the  other  laboratories  as  a  group. 

We  can  also  gain  some  insight  into  the  reported  Vgg  values  by  examining  some  descriptive 
statistics  obtained  from  each  laboratory.  As  a  measure  of  data  dispersion  and  variability, 
we  choose  the  standard  deviation  and  range,  which  are  displayed  in  Table  11.  For  our 
purposes,  the  range  was  calculated  by  subtracting  the  highest  V^g  value  from  the  lowest 
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Table  9:  Results  of  Kruskal  -  Wallis  Analysis 


Panel  Type 

Test  Type 

K  value 

Degrees  of 
Freedom 

Macrocomposite 

3  and  3 

44.97 

8 

Macrocomposite 

5  and  5 

43.05 

7 

Steel 

3  and  3 

48.17 

8 

Steel 

5  and  5 

47.15 

6 

Table  10:  K  Values  as  a  Function  of  Confidence  Level  and  Degrees  of  Freedom. 


Confidence 

Degrees  of  Freedom 

Level 

6 

7 

8 

90% 

K  >  10.64 

K  >  12.16 

K  >  13.36  ■ 

95% 

K  >  12.59 

K  >  14.07 

K  >  15.50 

99% 

K>  16.81 

K  >  18.47 

K  >  20.90 

Vjg  value  for  each  type  of  target  panel.  The  range  and  standard  deviation  of  the  data  for 
the  steel  are  most  noteworthy.  Although  a  certain  degree  of  stochastic  behavior  is  expected, 
the  range  of  the  steel  values  is  27%  of  the  mean  and  the  standard  deviation  is  10%  of 
the  mean.  Measures  of  such  magnitude  indicate  a  high  level  of  variability  and  dispersion 
in  the  reported  values.  Although  the  statistics  for  the  macrocomposite  panels  are  not 
as  dramatic,  we  conclude  that  there  is  sufficient  variability  to  question  site-to-site 
reproducibility  of  the  tests,  irrespective  of  the  material.  Note  that  it  is  possible  to  find  two 
or  three  laboratories  where  the  interlaboratory  agreement  is  very  good.  If  only  these  two 
or  three  laboratories  had  been  selected  to  participate  in  this  study,  a  completely  different 
conclusion  would  be  in  order.  This  fact  vividly  illustrates  the  necessity  for  widespread 
participation  in  any  interlaboratory  test  series. 

Discussion 

Fleuristics  and  experience  play  key  roles  in  many  fields  of  science,  but  clearly  their  presence 
effects  the  value  of  the  calculated  Ballistic  Protection  Limit  (PBL).  The  intuition  of  the 
test  engineer  influences  velocity  selection  and  hence  introduces  bias  into  the  outcome  of 
the  test.  This  concept  is  best  illustrated  by  considering  the  shot  sequence  data  of  Tables  1 
and  2  in  a  graphical  format,  as  displayed  in  Figure  7  for  the  steel  data  and  Figure  8  for  the 
macrocomposite  data.  For  most  of  the  laboratories,  once  the  test  engineer  obtained  a 
reversal — a  change  in  test  result  from  the  previous  test  shot — he  confined  his  attention  to 
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Table  11:  Descriptive  Statistics 


Statistic 

Steel 

V50  Value  (fps) 

Macrocomposite 
V50  Value  (fps) 

Mean 

1668 

1746 

Standard  Deviation 

166 

62.2 

Std.  Dev.  as  a  percentage  of  Mean 

10% 

4% 

Range 

447 

183 

Range  as  a  percentage  of  Mean 

27% 

11% 

a  relatively  narrow  velocity  range.  This  may  explain  why  there  is  usually  very  little 
difference  between  the  1  &  1  value  and  the  3  &  3  value.  For  the  steel  panels,  all  but 
one  laboratory  (Laboratory  6)  produced  a  1  &  1  value  within  100  fps  of  the  5  &  5 
value.  For  the  macrocomposite  panels,  three  laboratories  (Laboratories  2, 3,  and  9)  produced 
data  sets  with  wide  discrepancy  between  different  types  of  values.  Only  when  more 
data  points  are  forced  into  the  calculation  with  the  5  &  5  value  is  there  appreciable 
change  in  the  apparent  protection  ballistic  limit. 

This  may  be  due,  in  part,  to  overreliance  on  the  up  &  down  velocity  selection  recommended 
by  both  MIL-STD  662E  and  TOP  2-2-710.  As  a  point  of  information,  we  compiled 
information  from  Tables  3  and  4  presented  in  the  Results  section,  to  ascertain  how  often 
the  test  engineers  adjusted  the  velocity  in  increments  of  greater  than  100  fps.  Table  12 
shows  the  results  of  this  analysis.  This  table  confirms  that  most  test  engineers  complied 
with  up  &  down  shooting  with  modest  (<  100  fps)  increments  for  most  of  the  test  sequence. 
There  are,  however  two  notable  exceptions.  Laboratory  6  used  velocity  changes  in  excess 
of  100  fps  65%  of  the  time.  Laboratory  9  was  a  distant  second  with  42%  of  velocity  changes 
in  excess  of  100  fps.  For  the  macrocomposite  panels.  Laboratory  9  bypassed  the  lower 
value  for  the  macrocomposite  panels  by  using  a  400  fps  delta  to  adjust  velocity  from  one 
firing  to  the  next.  Although  this  may  seem  like  a  large  delta,  it  is  acceptable  under  both 
MIL-STD-662E  and  TOP  2-2-710. 

It  is  interesting  to  note  that  Laboratories  6  and  9  were  the  only  two  laboratories  that  reported 
large  numbers  of  complete  penetrations  below  1400  fps  for  the  steel  panels.  Rather  than 
treat  these  points  as  anomalous,  the  fact  that  two  independent  laboratories  obtained  this 
result  forces  us  to  consider  the  possibility  that  these  data  are  a  real  manifestation  of  this 
material's  behavior,  a  possible  transition  in  failure  phenomenon  at  low  impact  velocities. 
Here  again,  if  just  one  of  these  two  laboratories  had  not  participated  in  the  study  our 
outlook  on  this  data  would  be  much  different. 

A  possible  explanation  for  the  scatter  in  the  steel  data  may  be  a  manifestation  of  a  shatter 
gap.  The  steel  panels  used  for  this  study  had  a  hardness  of  approximately  50-52  HRC;  the 
projectiles,  a  hardness  of  approximately  61  HRC.  The  purpose  of  the  high  hardness  steel 
is  to  fracture  the  impacting  projectile  prior  to  penetration  of  the  target.  If  the  panel  is  not 
hard  enough,  the  intact  projectile  can  easily  defeat  the  panel  through  a  plugging  mechanism. 


Figure  8.  Recalculated  Values  for  Macrocomposite  Panels 


Table  12:  Velocity  Delta  Statistics 


Statistic 

Steel 

Macrocomposite 

Panels 

Panels 

Total  Shots 

228 

143 

Shots  with  delta  velocity  >  100  fps 

55 

41 

As  a  percentage 

24% 

29% 

Some  laboratories  observed  that  surface  hardness  on  some  of  the  steel  panels  fell  below 
the  expected  range  of  48  -  52  HRC.  A  uniformly  low  cross  sectional  hardness  would 
certainly  explain  low  penetration  velocities  for  the  steel  panels.  To  determine  if  this  was  a 
plausible  explanation,  microhardness  measurements  were  taken  on  the  cross  section  of 
some  of  the  panels. 

These  measurements  quantified  the  maximum  decarburized  layer  at  0.015  inch.  This  raises 
the  question:  can  a  0.015  inch  thick  soft  surface  layer  result  in  a  severe  performance  drop 
of  300  fps  in  an  armor  steel?  Although  this  question  has  not  been  studied  in  detail,  research 
conducted  at  ART  •  MD  did  show  that  removing  a  0.05  inch  layer  of  decarburization  resulted 
in  dramatic  improvement— almost  a  factor  of  two-in  a  ballistic  test.^^  Although  variable 
levels  of  decarburization  may  have  contributed  to  the  zone  of  mixed  results,  by  itself  it 
does  not  appear  sufficient  to  explain  the  magnitude  of  that  zone. 

Since  ballistic  testing  of  materials  is  intended  primarily  for  use  in  armor  design,  a  few 
observations  regarding  armor  design  are  in  order.  Historically,  lightweight  armor  has 
been  designed  to  meet  a  particular  value.^^  Therefore,  at  the  design  velocity  (V^  value), 
there  is  a  50%  probability  that  either  the  specified  design  projectile  or  fragments  will  cause 
damage  to  components  or  inflict  lethal  or  serious  injury  to  personnel.  Although  this  is  a 
good  first  approach  to  armor  design,  the  probability  of  survival  is  still  only  50%.  There  is 
a  potential  pitfall  when  evaluating  materials  strictly  based  on  the  value  alone. 

For  design  purposes,  a  material's  distribution  function  is  far  more  important  than  the 
value.  Consider  Figure  9,  which  shows  CDF  curves  for  three  different  normal  distributions. 
Although  the  associated  theoretical  armor  materials  have  equivalent  values,  they  do 
not  provide  equivalent  ballistic  protection  because  each  material  has  a  different  standard 
deviation  about  the  value. 

A  more  reasoned  approach  to  armor  design  should  involve  a  much  higher  probability  of 
survival,  in  excess  of  90%.  To  use  this  approach,  a  designer  must  know  more  about  the 
cumulative  distribution  function  which  describes  the  armor  performance  as  a  function  of 
velocity.  Given  the  resources  to  generate  sufficient  data,  the  distribution  function  can  be 
determined  by  recursive  modeling  techniques.  The  difficulty  with  this  approach  is  that 
the  number  of  individual  shots  required  to  demonstrate  a  high  survival  rate  within  an 
acceptable  confidence  level  is  a  function  of  the  CDF.  In  other  words,  there  is  no  statistical 
basis  for  standardizing  the  number  of  test  shots  required  to  adequately  determine  a  CDF 
suitable  for  design  purposes. 
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Figure  9.  Graph  Showing  Three  Normal  Cumulative  Distribution  Functions  with 


From  the  standpoint  of  a  statistically  reliable  test  design  too  few  panels  were  used  during 
this  study.  In  addition,  none  of  the  variables  reported  are  truly  independent.  However, 
the  number  of  panels  used  for  this  study  was  far  in  excess  of  that  normally  tested  to  establish 
a  Vgg  PEL.  In  fact,  the  number  of  panels  used  for  these  tests  was  between  two  and  four 
times  greater  than  a  typical  compliment  of  panels.  To  apply  traditional  statistical  methods, 
the  number  of  data  points  required  is  usually  greater  than  thirty  (30),  a  twofold  increase 
over  the  number  of  shots  traditionally  used  for  these  tests.  The  net  impact  of  requiring 
more  than  thirty  shots  for  determination  of  a  value  would  be  a  materials  cost  increase 
of  between  four  (4)  and  eight  (8)  times  existing  costs  and  commensurate  cost  increases  for 
ballistic  testing.  Moreover,  the  existing  MIL-STD-662E  and  TOP  2-2-710  provide  the  test 
engineer  with  a  great  deal  of  latitude  in  selecting  the  sequence  of  firing  velocities. 

Prior  to  drawing  final  conclusions  based  on  this  statistical  analysis,  we  must  reconsider 
our  original  assumptions: 

(1)  the  target  panels  have  identical  microstructures  and  physical  properties, 

(2)  the  AP  M2  projectiles  are  essentially  identical, 

(3)  any  excessive  variahon  in  the  test  results  can  be  attributed  to  subjective  test  methodology. 

Since  all  the  steel  panels  and  similarly  all  the  ceramic  composite  panels  were  each  fabricated 
at  one  facility,  undergoing  the  same  processing  treatments,  we  believe  assumption  one 
remains  valid  within  some  negligible  tolerance  range.  A  similar  argument  holds  true  for 
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the  projectiles.  Assumption  three  states  that  any  'large'  variation  must  result  from 
differences  in  the  test  methodology  used.  Specifically  the  variance  introduced  by  using 
different,  subjective  velocity  selection  procedures  and  test  hardware  (including  systematic 
differences  in  fixtures,  distance  and  time  measuring  equipment,  environmental  conditions, 
and  human  error)  additively  combine  to  form  the  overall  variance  in  the  result.  We  continue 
to  support  the  assumption  that  any  large  variations  in  test  results  observed  during  these 
interlaboratory  tests  is  a  result  of  non-standardized  test  methodology,  specifically  the 
absence  of  conventional  testing  protocols  and  test  hardware  configurations. 

The  Tukey  and  Kruskal  -  Wallis  analyses  were  both  used  to  compare  the  standardized 
values.  Both  of  these  analyses  and  the  descriptive  statistics,  calculated  to  compare  the 
reported  values,  indicate  that  the  test  results  obtained  from  the  laboratories  are  not 
reproducible.  The  analysis  produced  the  same  result  regardless  of  the  type  of  value 
considered.  Existing  statistical  differences  exceed  those  that  are  reasonably  expected  to 
occur  as  a  result  of  any  small  variations  in  the  panels  or  projectiles.  We  surmise  that 
ballistic  tests  conducted  in  accordance  with  MIL-STD-662E  are  not  reproducible  because 
of  the  different  test  methodologies  and  test  hardware  indigenous  to  each  participating 
laboratory. 

It  is  critical  to  note  that  the  test  methodologies  in  question  include: 

(1)  the  subjective  method  used  to  select  firing  velocities,  and 

(2)  the  configuration  of  test  hardware  including  fixtures. 

These  variables  were  under  the  control  of  each  laboratory  within  the  guidelines  of  MIL- 
STD-662E.  Each  laboratory  participating  in  the  interlaboratory  study  conducted  their 
ballistic  tests  in  accordance  with  MIL-STD-662E,  as  witnessed  by  the  Project  Manager. 
Bearing  these  facts  in  mind,  we  find  it  reasonable  to  conclude  that  unacceptably  high 
variations  in  the  test  results,  which  show  that  the  tests  are  not  reproducible,  indicates  that 
the  latitudes  provided  by  the  test  standards  outlined  in  MIL-STD-662E  can  only  be 
overcome  with  improvements  to  ballistic  testing  specifications. 

It  is  interesting  to  note  that  the  current  version  of  MIL-STD-662,  the  E  version,  has  less 
demanding  requirements  for  calculating  a  value  than  some  previous  versions.  Table 
13  includes  a  listing  of  each  version  of  the  standard  together  with  the  requirements  for 
calculating  a  V^g  value.  In  particular,  note  that  versions  A,  B,  and  C  required  10  impacts  as 
a  minimum.  In  addition,  standard  B  specified  that  a  zone  of  mixed  results  in  excess  of  150 
fps  required  additional  data  (shots)  to  further  elucidate  the  behavior  of  the  armor  material. 
Although  the  number  of  shots  implicit  in  this  requirement  may  seem  excessive  to  those 
well  practiced  in  the  art  of  ballistic  testing,  the  results  obtained  from  this  study  indicate 
that  even  five  partial  penetrations  and  five  complete  penetrations  are  not  sufficient  to 
guarantee  reproducibility  of  interlaboratory  test  results.  Based  upon  the  data  obtained 
during  the  interlaboratory  tests  and  our  analysis  of  that  data,  we  present  the  following 
conclusions  and  recommendations: 
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Conclusions 

1 .  All  of  the  laboratories  calculated  V50  values  according  to  MIL-STD-662E,  TOP  2-2-710, 
or  both. 

2.  Using  a  95%  confidence  level,  ballistic  test  results  are  not  reproducible  from  one 
laboratory  to  the  next  for  both  steel  and  macrocomposite  panels. 

3.  Reducing  the  confidence  level  by  increments  of  5%,  10%,  or  15%  (to  90%,  85%,  or  80%) 
does  not  alter  the  foregoing  conclusion. 

4.  MIL-STD-662E  does  not  effect  reproducible  ballistic  test  results  for  the  materials  studied: 
the  procedures  set  forth  in  the  standard  need  to  be  revised  to  make  interlaboratory 
ballistic  data  reproducible. 

Recommendations 

1.  Establish  stricter  rules  for  velocity  selection,  perhaps  by  using  a  velocity  selection 
algorithm.  This  would  help  to  eliminate  subjective  bias. 

2.  Standardize  specimen  test  sizes  and  fixturing  requirements. 

3.  Increase  the  minimum  number  of  shots  required  to  calculate  a  V50  value.  Older  versions 
of  MIL-STD-662E  required  five  partial  penetrations  and  five  complete  penetrations  to 
calculate  a  valid  V50  value. 

4.  Establish  a  technical  committee  through  a  national,  not-for-profit  organization  such  as 
the  American  Society  for  Testing  and  Materials  ( ASTM)  to  tighten  specifications,  ensure 
statistical  confidence,  choose  velocity  selection  algorithms,  and  measure  reproducibility. 
By  virtue  of  forming  under  ASTM,  the  committee  would  be  open  to  membership  from 
Government,  Academia,  and  Industry. 
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Table  13:  Chronology  of  MIL-STD-662. 


o 

Z 


* 


TI 


A  copy  of  this  standard  was  not  available  for  review. 

The  selection  of  the  number  of  impacts  and  velocity  spread  is  directed  by  the  applicable  procurement  specification,  the  contracting 
officer,  or  the  test  engineer. 
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Minitab  Output  Comparing  the  V50  Values  Calculated  Using  the  JTCG/AS 
Interlaboratory  Test  Data  in  Accordance  with  TOP  2-2-710 

Table  A1 

Steel  V50  Values  with  1  Partial  Penetration  and  1  Complete  Penetration  in  50  fps 


INDIVIDUAL  95  PCT  Cl’S  FOR  V50 

BASED  ON  POOLED  STDEV 


LA 

1 

B  SHOTS 

2 

V50 

1812.5 

STDEV 

34 . 6 

- + - 

^ - 

- + 

(- 

2 

2 

1870.0 

12.7 

3 

2 

1807.0 

8.5  ■ 

(-* 

4 

2 

1656.5 

27.6 

(-*-) 

5 

2 

1595.0 

32.5 

{-*-) 

6 

2 

1258.0 

25.5 

{-*-) 

7 

2 

1837 . 5 

33.2 

( 

8 

2 

1583.5 

20,5 

(-*-) 

9 

2 

1448  -  5 

21.9 

(-*-) 

+ 

POOLED 

STDEV  = 

25.6 

1400 

1600 

1800 

TABLE  OF  PAIRWISE  COMPARISONS 

1 

2 

3 

4 

5 

6 

7 

2 

-158.9 

43.9 

3 

-95.9 

106.9 

-38.4 

164.4 

4 

54.6 

257 . 4 

112.1 

314.9 

49.1 
251 . 9 

5 

116.1 

318.9 

173 . 6 
376.4 

110 . 6 
313.4 

-39.9 

162.9 

6 

453.1 

655.9 

510.6 

713.4 

447 . 6 
650.4 

297.1 

499.9 

235.6 

438.4 

7 

-126.4 

76.4 

-68.9 

133.9 

-131.9 
70 . 9 

-282.4 

-79.6 

-343.9 

-141.1 

-680.9 

-478.1 

8 

127 . 6 
330.4 

185.1 

387.9 

122.1 

324.9 

-28.4 

174.4 

-89.9 

112.9 

-426.9 
-224 . 1 

152 . 6 
355.4 

9 

262.6 

465.4 

320.1 

522.9 

257.1 

459.9 

106.6 

309.4 

45.1 

247.9 

-291.9 

-89.1 

287.6 

490.4 

Tukey ' s  pairwise  comparisons 


Family  error  rate  =  0.0500 
Individual  error  rate  =  0.00336 
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Appendix 


Minitab  Output  Comparing  the  V50  Values  Calculated  Using  the  JTCG/AS 
Interlaboratory  Test  Data  in  Accordance  with  TOP  2-2-710 

Table  A2 

Steel  V50  Values  with  2  Partial  Penetrations  and  2  Complete  Penetrations  in  60  fps 


INDIVIDUAL  95  PCT  Cl'S  FOR  V50 

BASED  ON  POOLED  STDEV 


LAB 

SHOTS 

V50 

STDEV 

- ^ - 

- + - 

- + - 

1 

4 

1795.0 

11 . 5 

(-*) 

2 

4 

1851.5 

22.6 

(*-) 

3 

4 

1823.0 

22,3 

(-*) 

4 

4 

1684.2 

20.1 

{*-) 

5 

4 

1562.0 

14.0 

(*) 

6 

4 

1676.8 

19.7 

(-*) 

7 

4 

1791.2 

21.5 

(*-) 

8 

4 

1597.7 

20.3 

( 

-*) 

9 

4 

1429.3 

23.5 

(*-} 

- + - 

- + - 

- + - 

POOLED  STDEV  = 

19.9 

1500 

1650 

1800 

TABLE  OF  PAIRWISE  COMPARISONS 


1 

2 

3 

4 

5 

6 

7 

8 

2 

•  -103,8 
-9.2 

3 

-75.3 

19.3 

-18.8 

75.8 

4 

63 . 5 
158.0 

120.0 

214 . 5 

91.5 

186.0 

5 

185 . 7 
280.3 

242.2 

336.8 

213,7 

308.3 

75.0 

169.5 

6 

71 . 0 
165.5 

127.5 

222.0 

99.0 

193 . 5 

-39.8 

54.8 

-162.0 

-67.5 

7 

-43.5 

51.0 

13 , 0 
107.5 

-15.5 

79.0 

-154.3 

-59.7 

-276 . 5 
-182.0 

-161.8 

-67.2 

8 

150.0 

244 . 5 

206 . 5 
301.0 

178.0 

272.5 

39.2 

133.8 

-83.0 

11.5 

31.7 

126.3 

146.2 

240.8 

9 

318.5 

413.0 

375.0 

469.5 

346.5 

441.0 

207.7 

302,3 

85.5 

180.0 

200.2 

294.8 

314.7 

409.3 

121.2 

215,8 

Tukey's  pairwise  comparisons 

Family  error  rate  =  0.0500 
Individual  error  rate  =  0.00231 
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Minitab  Output  Comparing  the  V50  Values  Calculated  Using  the  JTCG/AS 
Interlaboratory  Test  Data  in  Accordance  with  TOP  2-2-710 

Table  A3 

Steel  V50  Values  with  3  Partial  Penetrations  and  3  Complete  Penetrations  in  150  fps 


LAB 

SHOTS 

V50 

STDEV 

1 

6 

1773.3 

44.9 

2 

6 

1850.0 

33.5 

3 

6 

1877.8 

47.3 

4 

6 

1657.0 

57.4 

5 

6 

1556.3 

48.2 

6 

6 

1643.7 

55.2 

7 

6 

1793.7 

35 . 1 

8 

6 

1601,7 

22.5 

9 

6 

1450.2 

41.2 

POOLED  STDEV  = 

44.1 

INDIVIDUAL  95  PCT  Cl'S  FOR  V50 

BASED  ON  POOLED  STDEV 

- 4. - + - ^ - ^ 


- 4. - ^ - - ^ 

1500  1650  1800  1950 


TABLE  OF  PAIRWISE  COMPARISONS 


1 

2 

3 

4 

2 

-159.6 

6,3 

3 

-187,4 

-21,6 

-110.8 

55.1 

4 

33.4 

199.3 

110.1 

275 . 9 

137.9 

303.8 

5 

134 . 1 
299.9 

210 . 7 
376.6 

238.6 

404.4 

17.7 

183.6 

6 

46.7 

212.6 

123 . 4 
289.3 

151.2 

317.1 

-69 . 6 
96 . 3 

7 

-103  .3 
62.6 

-26.6 

139.3 

1.2 

167 . 1 

-219 . 6 
-53.7 

8 

88,7 

254.6 

165.4 

331.3 

193.2 

359.1 

-27 . 6 
138.3 

9 

240.2 

406.1 

316.9 

482.8 

344.7 

510.6 

123 . 9 
289.8 

Tukey’s  pairwise  comparisons 


5  6  7  8 


-170 . 3 
-A  .4 


-320.3 

-232.9 

-154  .  4 

-67.1 

-128 . 3 

-40 . 9 

109 . 1 

37.6 

124.9 

274.9 

23.2 

110,6 

260.6 

68.6 

189.1 

276.4 

426.4 

234.4 

Family  error  rate  =  0.0500 
Individual  error  rate  =  0.00214 
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Appendix 


Minitab  Output  Comparing  the  V50  Values  Calculated  Using  the  JTCG/AS 
Interlaboratory  Test  Data  in  Accordance  with  TOP  2-2-710 

Table  A4 

Steel  V50  Values  with  5  Partial  Penetrations  and  5  Complete  Penetrations 


INDIVIDUAL  95  PCT  Cl’S  FOR  V50 

BASED  ON  POOLED  STDEV 


LAB 

SHOTS 

V50 

STDEV 

- ^ - + - + - 

1 

10 

1814 . 5 

46.6 

2 

10 

1834.0 

118.2 

4 

10 

1693 . 1 

46.0 

5 

10 

1503.3 

53.7 

6 

10 

1513.8 

192.8 

7 

10 

1745.8 

96.0 

(---* - ) 

8 

10 

1585.3 

53 . 0 

( - *---) 

- + - +  -- - + - 

POOLED  STDEV  = 

100.2 

1500  1650  1800  195 

TABLE 

OF  PAIRWISE  COMPARISONS 

1 

2 

4 

5 

6 

7 

2 

-156 

117 

4 

-15 

4 

258 

278 

5 

175 

194 

53 

448 

467 

326 

6 

164 

184 

43 

-147 

437 

457 

316 

126 

7 

-68 

-48 

-189 

-379 

•  -369 

205 

225 

84 

-106 

-95 

8 

93 

112 

-29 

-219 

-208 

24 

366 

385 

244 

55 

65 

297 

Tukey ' s 

pairwise 

comparisons 

Family  error  rate  -  0.0500 
Individual  error  rate  =  0.00338 
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Appendix 


Minitab  Output  Comparing  the  V50  Values  Calculated  Using  the  JTCG/AS 
Interlaboratory  Test  Data  in  Accordance  with  TOP  2-2-710 


Table  A5 

Ceramic  V50  Values  with  1  Partial  Penetration  and  1  Complete  Penetration  in  50  fps 


LAB 

SHOTS 

V50 

STDEV 

INDIVIDUAL  95  PCT  Cl'S  FOR  V50 

BASED  ON  POOLED  STDEV 

1 

2 

1703.5 

33.2 

2 

2 

1896.0 

4.2 

3 

2 

1601.5 

9.2 

4 

2 

1638.5 

19.1 

5 

2 

1755.0 

17.0 

(--*-) 

6 

2 

1876.0 

7.1 

7 

2 

1657.5 

21  .  9 

8 

2 

1671.0 

25.5 

9 

2 

1673.0 

7.1 

POOLED  STDEV  = 

18.5 

1600  1700  1800  1900 

TABLE  OF  PAIRWISE  COMPARISONS 


1 

2 

3 

4 

5 

6 

7 

2 

-265.8 

-119.2 

3 

28.7 

175.3 

221,2 

367.8 

4 

-8.3 

138.3 

184.2 

330.8 

-110.3 

36.3 

5 

-124.8 

21.8 

67.7 

214.3 

-226.8 

-80.2 

-189.8 

-43.2 

6 

-245,8 

-99.2 

-53.3 

93.3 

-347.8 

-201.2 

-310.8 

-164.2 

-194.3 

-47.7 

7 

-27,3 

119.3 

165.2 

311.8 

-129.3 

17.3 

-92.3 

54.3 

24.2 

170.8 

145.2 

291.8 

8 

-40.8 

105.8 

151.7 

298.3 

-142 . 8 
3.8 

-105,8 

40.8 

10 , 7 
157.3 

131.7 

278.3 

-86.8 

59.8 

9 

-42.8 

103.8 

149.7 

296.3 

-144.8 

1.8 

-107.8 

38.8 

8.7 

155.3 

129.7 

276.3 

-88.8 

57.8 

Tukey 

'8  pairwise 

comparisons 

Family  error  rate  =  0.0500 
Individual  error  rate  =  0.00336 
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Appendix 


Minitab  Output  Comparing  the  V50  Values  Calculated  Using  the  JTCG/AS 
Interlaboratory  Test  Data  in  Accordance  with  TOP  2-2-710 

Table  A6 

Ceramic  V50  Values  with  2  Partial  Penetrations  and  2  Complete  Penetrations  in  60  fps 


INDIVIDUAL  95  PCT  Cl'S  FOR  V50 

BASED  ON  POOLED  STDEV 


LAB 

SHOTS 

V50 

STDEV 

- + - 

- + - 

- -f- - 

1 

4 

1747 . 0 

22.0 

2 

4 

1781 . 0 

11.5 

-) 

3 

4 

1771.7 

21.5 

4 

4 

1683.7 

24.7 

5 

4 

1754.5 

24.1 

6 

4 

1896.7 

25.5 

7 

4 

1652.2 

26.8 

8 

4 

1675.3 

23.4 

9 

4 

1688.5 

25.7 

- + - 

- + - 

- + - 

POOLED  STDEV  = 

23.2 

1680 

1760 

1840 

TABLE  OF  PAIRWISE  COMPARISONS 


12  3  4 


2  -89.2 

21.2 


3 

-80. 

.0 

-46 

.  0 

30, 

.5 

64 

.  5 

4 

8  . 

.  0 

4  2 

.0 

32  . 

.  8 

118, 

.5 

152 

.5 

143  . 

.2 

5 

-62  , 

.  7 

-28 

.  7 

-38. 

.  0 

-126  . 

.0 

47. 

.7 

81 

.7 

72  . 

,  5 

-15, 

,  5 

6 

-205  . 

.  0 

-171 , 

.0 

-180  . 

,2 

-268  . 

,2 

-94  , 

.  5 

-60 

.  5 

-69, 

.  8 

-157  . 

,  8 

7 

39  . 

.  5 

73  . 

.  5 

64  . 

.3 

-23  . 

.  7 

150. 

.0 

184. 

.0 

174  . 

,  7 

86, 

.  7 

8 

16  . 

.  5 

50. 

.  5 

41 , 

.3 

-46, 

.7 

127, 

.0 

161, 

.0 

151 . 

,  7 

63  . 

.7 

9 

3, 

.3 

37 

.3 

28, 

.0 

-60. 

.0 

113, 

,  7 

147. 

.7 

138. 

5 

50, 

,  5 

6  7  8 


-197 . 5 
-87 . 0 


47  . 

.0 

189 

.3 

157. 

.5 

299, 

.7 

24  , 

.  0 

166 

.3 

-78  . 

.  2 

134, 

.5 

276  , 

.  7 

32  , 

.2 

10. 

.  8 

153 

.  0 

-91, 

.  5 

-68  . 

.5 

121 . 

2 

263, 

,  5 

19. 

.0 

42  , 

.0 

Tukey's  pairwise  comparisons 

Family  error  rate  =  0.0500 
Individual  error  rate  -  0.00231 
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Minitab  Output  Comparing  the  V50  Values  Calculated  Using  the  JTCG/AS 
Interlaboratory  Test  Data  in  Accordance  with  TOP  2-2-710 

Table  A7 

Ceramic  V50  Values  with  3  Partial  Penetrations  and  3  Complete  Penetrations  in  150  fps 


INDIVIDUAL  95  PCT  Cl'S  FOR  V50 

BASED  ON  POOLED  STDEV 


LAB 

SHOTS 

V50 

STDEV  -  + - + - + - + - 

1 

6 

1741.3 

21.1 

2 

6 

1845.0 

54.6 

3 

6 

1804.3 

48.5 

4 

6 

1694.5 

28.6 

5 

6 

1760.8 

43.7 

6 

6 

1906.0 

49.7 

7 

6 

1634.0 

53.8 

8 

6 

1669.2 

39.8 

9 

6 

1692.3 

4  6.1  (--* - ) 

+  -t-  4-  -h 

POOLED  STDEV  = 

44  .  2 

1600  1700  1800  1900 

TABLE  OF  PAIRWISE  COMPARISONS 


1 

2 

3 

4 

5 

6 

7 

8 

2 

-186 . 9 
-20.5 

3 

-146 . 2 
20.2 

-42.5 

123.9 

4 

-36.4 

130.0 

67.3 

233.7 

26.6 
193 . 0 

5 

-102.7 

63.7 

1.0 

167.4 

-39.7 

126.7 

-149.5 

16.9 

6 

-247.9 

-81.5 

-144.2 

22.2 

-184.9 

-18.5 

-294.7 

-128.3 

-228.4 
-62 . 0 

7 

24 . 1 
190.5 

127 . 8 

294 .2 

87 . 1 
253.5 

-22 . 7 
143.7 

43 . 6 
210.0 

188 . 8 
355.2 

8 

-11.0 
155 . 4 

92 . 6 
259.0 

52 . 0 
218.4 

-57.9 

108.5 

8 . 5 

174 . 9 

153 . 6 
320.0 

-118.4 

48 . 0 

9 

-34,2 

132.2 

69 . 5 
235.9 

28.8 

195.2 

-81 . 0 
85.4 

-14.7 

151.7 

130.5 

296.9 

-141 . 5 
24.9 

-106.4 

60.0 

Tukey '  s 

pairwise 

comparisons 

Family  error  rate  -  0.0500 
Individual  error  rate  =  0,00214 
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Minitab  Output  Comparing  the  V50  Values  Calculated  Using  the  JTCG/AS 
Interlaboratory  Test  Data  in  Accordance  with  TOP  2-2-710 

Table  A8 

Ceramic  V50  Values  with  5  Partial  Penetrations  and  5  Complete  Penetrations 


INDIVIDUAL  95  PCT  Cl’S  FOR  V50 

BASED  ON  POOLED  STDEV 

LAB  SHOTS  V5  0  STDEV  - + - + - +  - 


1  10  1760.1  39.2  ( - * - ) 

2  10  1974.6  310.4  ( - *--- 

4  10  1705.7  47.2  ( - * - ) 

5  10  1757.9  39.6  { - * - } 

6  10  1872.2  121.1  ( - * - ) 

7  10  1656.6  33.4  { - * - ) 

8  10  1710.9  43.6  ( - * - } 

9  10  1908,1  378.4  ( - * - } 


POOLED  STDEV  =  181.2  1650  1800  1950 


TABLE  OF  PAIRWISE  COMPARISONS 

1  2  4  5  6  7  8 


2  -468 

39 


4 

-199 

16 

308 

522 

5 

-251 

-37 

-305 

255 

470 

201 

6 

-365 

-151 

-420 

141 

356 

87 

7 

-150 

65 

-204 

357 

571 

302 

8 

-204 

10 

-258 

302 

517 

248 

9 

-401 

-187 

-456 

105 

320 

51 

Tukey's  pairwise  comparisons 

Family  error  rate  =  0.0500 
Individual  error  rate  =  0.00258 


-368 

139 


152 

-38 

355 

469 

206 

-92 

-308 

300 

415 

199 

403 

-289 

-505 

-450 

103 

217 

2 

56 

35 
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VA;  Mr.  Earl  Wilhelm  of  the  Boeing  Defense  &  Space  Group;  Dr.  Barry  E.  Bodt  of  the  U.S. 
Army  Research  Laboratory  Advanced  Computational  &  Information  Sciences  Directorate; 
Mr.  William  A.  Gooch  Jr.  and  Mr.  Edward  J.  Rapacki  of  the  U.S.  Army  Research  Laboratory, 
Weapons  Technology  Directorate;  Dr.  Eric  B.  Kula,  Dr.  Martin  G.H.  Wells,  Dr.  Ralph  RI. 
Adler,  Dr.  Jonathan  S.  Montgomery,  Dr.  Joseph  Prifti,  Dr.  Albert  Chang,  Mr.  Albert  A. 
Anctil,  Mr.  William  E.  Haskell  III,  and  Mr.  James  Mackiewicz  of  the  U.S.  Army  Research 
Laboratory  Materials  Directorate;  Mr.  Ellory  Sanderson  and  Mr.  Todd  M.  Wagner  of  the 
U.S.  Army  Combat  Systems  Test  Activity;  Dr.  Mark  Vangel  of  the  National  Institute  for 
Standards  and  Technology;  Dr.  Marc  A.  Adams  and  Dr.  James  G.  Zwissler  of  the  Jet 
Propulsion  Laboratory,  California  Institute  of  Technology. 
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1  ATTN:  R.  Newcomer,  Mail  Code  1021310 

Douglas  Aircraft  Company,  McDonnell  Douglas  Corporation,  3855  Lakewood  Boulevard,  Long 
Beach,  CA  90846 

1  ATTN:  Chief  of  Materials  and  Processes 

1  ATTN:  Mr.  A.R.  Parker,  Mail  Code  35-95 

Northrop  Corporation,  Aircraft  Division,  1  Northrop  Avenue,  Hawthorne,  CA  90250-3277 
1  ATTN:  M.  Tommel,  3872-62 

1  ATTN:  Chief  of  Materials  and  Processes 

Rockwell  International  Corporation,  North  American  Aircraft  Division,  P.O.  Box  92098,  Los 
Angeles,  CA  90009 

1  ATTN:  Chief  of  Materials  and  Processes 

General  Dynamics,  Fort  Worth  Division,  P.O.  Box  748,  Fort  Worth,  TX  67101 
1  ATTN:  Chief  of  Materials  and  Processes 
1  ATTN:  Susan  Kiehl,  MZ  2853 

General  Dynamics,  Convair  Aerospace  Division,  P.O.  Box  748,  Fort  Worth,  TX  76101 
1  ATTN:  Mfg.  Engineering  Technical  Library 

Sikorsky  Aircraft,  6900  Main  Street,  Stratford,  CT  06601-1381 
1  ATTN:  Chief  of  Materials  and  Processes 

1  ATTN:  T.  Murphy,  Mail  Stop  S312A 

1  ATTN:  J.  David  Pansier,  Mail  Stop  ZIOIA 

1  ATTN:  Gerald  Burblis,  Mail  Stop  ZlOl  A 

1  ATTN:  Ronald  Dexter,  Mail  Stop  ZIOIA 

Textron  Lycoming,  550  South  Main  Street,  Stratford,  CT  06497-2452 
1  ATTN:  V  Nangia,  Manager,  Advanced  Materials  Technology  Laboratory 

1  Dianne  Chong,  McDonnell  Douglas  Missile  Systems,  M/C  1063246,  P.O.  Box  516,  St.  Louis,  MO 
63166-0516 
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Vought  Corporation,  RO.  Box  226144,  Dallas,  TX  75222 
1  ATTN:  Chief  of  Materials  and  Processes 
1  ATTN:  D.  Peterson,  Advanced  Technical  Center 

Grumman  Aerospace  Corporation,  Bethpage,  NY  11714 
1  ATTN:  Chief  of  Materials  and  Processes 

1  ATTN:  R  Adler,  Mail  Stop  A02-26 

1  ATTN:  S.  Demay,  PL12,  Dept.  441 
1  ATTN:  J.  Kennedy,  Mail  Stop  A02-26 
1  ATTN:  R  Shaw,  Mail  Stop  A04-12 

1  ATTN:  J.  Greenspan,  Mail  Stop,  A04-12 

1  ATTN:  L.  R.  Bohanic 

Lockheed  Aeronautical  Systems  Company,  86  South  Cobb  Drive,  Zone  150,  Marietta,  GA  30063- 
0199 

1  ATTN:  D.  Richardson,  Dept.  73-C2  (Zone  0199) 

1  ATTN:  D.  Chellman 

1  ATTN:  Chief  of  Materials  and  Processes 

1  ATTN:  J.  Whitehead 

1  Mr.  Martin  E.  Smirlock,  Foster  Miller,  Incorporated,  350  Second  Avenue,  Waltham,  MA  02154- 
1196 

Lawrence  Livermore  National  Laboratory,  University  of  California,  Chemistry  and  Materials 
Science  Department,  RO.  Box  808,  L-369,  Livermore,  CA  94551 
1  ATTN:  Mr.  Thomas  E.  Shell 

Creusot-Marrel,  Incorporated,  724  West  Lancaster  Avenue,  Suite  213,  Wayne,  PA  19087 
1  ATTN:  Mr.  Billy  F.  Holcombe 

The  Survice  Engineering  Company,  1003  Old  Philadelphia  Road,  Suite  103,  Aberdeen,  MD  21001 
1  ATTN:  Mr.  Ronald  B.  Levy 

1  ATTN:  Mr.  Greg  Thomspon 

1  ATTN:  Mr.  Dyrk  Van  Dusen 

1  Mr.  Richard  Palicka,  Cercom,  Inc.,  1960  Watson  Way,  RO.  Box  70,  Vista,  CA  92083 

Carborundum,  RO.  Box  1054,  Niagara  Falls,  NY  14302-1054 
1  ATTN:  Mr.  John  C,  Ruppel 

1  ATTN:  Mr.  Robert  A.  Palia 

1  Mr.  Dale  B.  Atkinson,  7703  Carrleigh  Parkway,  Springfield,  VA  22152 

1  Mr.  Patrick  Zabel,  Route  1,  Box  397D,  Devine,  TX  78016 

Allied  Signal  Corporation,  Armor  Business  Group,  PO  Box  31,  Petersburg,  VA  23804 
1  ATTN:  Mr.  Dennis  Lang 


47 


No.  of 
Copies 


Distribution  List 


To 


Allied  Signal  Corporation,  Technical  Contracting,  PO  Box  1021,  Morristown,  NJ  07962-1021 
1  ATTN:  Mr,  Martin  P.  Chobrda 

The  Analytic  Sciences  Corporation,  Florida  Office,  1992  Lewis  Turner  Boulevard,  Fort  Walton 
Beach,  FL  32547 
1  ATTN:  Ms.  Combs 

Titan  Research  &  Technology,  5117  Johnson  Drive,  Pleasanton,  CA  94588 
1  ATTN:  Mr.  Dennis  L.  Orphal 

1  Mr.  Walter  S.  Vikestad,  3007  Rolling  Green  Drive,  Churchville,  MD  21028 

1  James  Robinson,  Ballistic  Technology  Inc.,  23922  Broadhorn  Drive,  Laguna  Niguel,  CA  92677 

1  N.  Albert  Moussa,  BlazeTech  Corporation,  21  Erie  Street,  Cambridge,  MA  02139 

University  of  Texas  at  Austin,  Institute  for  Advanced  Technology,  4030-2  West  Braker  Lane,  Austin, 
TX  78759-5329 
1  ATTN:  Mr.  Bless 

University  of  Denver  Colorado  Seminary,  Denver  Research  Institute,  PO  Box  100758,  Denver, 
CO  80250 

1  ATTN:  Mr.  Ullyatt  (BW  227) 

Booz*  Allen  &  Hamilton,  Inc.  4141  Col  Glenn  Highway,  Suite  131,  Wright-Patterson  Air  Force 
Base,  OH,  45433-7542 
1  ATTN:  Mr.  John  M.  Vice 

Director,  Survivability/Vulnerability  Information  Analysis  Center,  2130  Eighth  Street  STEl, 
Wright-Patterson  Air  JForce  Base,  OH,  45433-7542 
1  ATTN:  Director 

1  Dr.  John  G.  Cowie,  70  Isleta  Drive,  White  Rock,  NM  87544 

1  Center  for  Naval  Analysis,  4401  Fort  Avenue,  PO.  Box  16268,  Alexandria,  VA  22302-0268 

Jet  Propulsion  Laboratory,  California  Institute  of  Technology,  4800  Oak  Grove  Drive,  Pasadena, 
CA  91109 

1  ATTN:  Dr.  M.  Adams 

1  ATTN:  Dr.  J.  Zwissler 

Simula  Government  Products,  Incorporated,  10016  S.  51st  Street,  Phoenix,  AZ  85044 
1  ATTN:  Mr.  F.  Stan  Lyons 

1  ATTN:  Mr.  Chris  A.  Bradney 

1  ATTN:  Mr.  William  J.  Perciballi 

H.P.  White  Laboratory,  Incorporated,  3114  Scarboro  Road,  Street,  MD  21154-1822 
1  ATTN:  Mr.  Craig  B,  Dunn 

1  ATTN:  Mr.  Eric  Dunn 
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1  ATTN:  Ms.  Suzanne  A.  Royal 

1  ATTN:  Mr,  Bruce  L.  Morris 

1  ATTN:  Mr.  John  R  Riegei 

1  ATTN:  Dr.  Charles  E,  Anderson,  Jr. 

Ceradyne,  Incorporated,  3169  Redhill  Avenue,  Costa  Mesa,  CA  92626 
1  ATTN:  Mr.  Joel  P.  Moscowitz 

1  ATTN:  Mr.  David  Reed 

1  ATTN:  Mr.  John  P.  Bird 

1  ATTN:  Mr.  John  Walsh 

Ceradyne,  Incorporated,  P.O.  Box  925,  3449  Church  Street,  Scottdale,  GA  30079 
1  ATTN:  Mr.  Earl  Conabee 

1  Mr.  Albert  A.  Anctil,  9  Harris  Drive,  Burlington,  MA  01803  , 

1  Dr.  Morris  Azrin,  107  Manning  Street,  Needham,  MA  02194  ^ 

5  Mr.  John  H.  Graves  (Author),  10  Middlebury  Lane,  Beverly,  MA  01915-1358 
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